An image acquired by capturing an image of an object through an image pickup apparatus such as a digital camera contains an image blur component which is an image deterioration component caused by spherical aberration, comatic aberration, field curvature, astigmatism or the like of an image taking optical system (hereinafter simply referred to as “optical system”). Such an image blur component is generated because a light flux emitted from one point of the object forms an image with some divergence on an image pickup surface, the light flux being normally converged at one point when there is no aberration or diffraction.
Such an image blur component is optically represented by a point spread function (PSF), and different from a blur caused by defocusing. Color blur in a color image caused by longitudinal chromatic aberration, chromatic spherical aberration, or chromatic comatic aberration of the optical system can be said to be a difference between blurring degrees of respective wavelengths of light.
As a method for correcting an image blur component, there is known a correction method which uses information of an optical transfer function (OTF) of an optical system. This method is referred to as “image restoration”. Hereinafter, processing for correcting (reducing) an image blur component by using the information of the optical transfer function (OTF) of the optical system is referred to as “image restoration processing”.
The outline of the image restoration processing is as follows.
When g(x, y) represents a deteriorated image (input image) containing an image blur component, f(x, y) represents an original non-deteriorated image, h(x, y) represents a point spread function (PSF) which is a Fourier pair of an optical transfer function, * represents convolution, and (x, y) represents coordinates on an image, the following expression is established:g(x, y)=h(x, y)*f(x, y).
When converting the above expression into a form of a two-dimensional frequency surface through Fourier transformation, the expression is presented as follows, which takes a form of a product for each frequency:G(u, v)=H(u, v)·F(u, v)where H indicates a result of Fourier transformation of the point spread function (PSF), in other words, an optical transfer function (OTF), and (u, v) indicates coordinates on the two-dimensional frequency surface, in other words, a frequency.
In order to acquire the original image from the deteriorated image, both sides of the expression only need to be divided by H as below:G(u, v)/H(u, v)=F(u, v).
Returning the F(u, v) through inverse Fourier transformation to a real surface enables acquisition of a restored image equivalent to the original image f(x, y).
When R represents a result of inverse Fourier transformation of H−1, performing convolution processing for an image in the real surface as represented by the following expression similarly enables acquisition of the original image:g(x, y)*R(x, y)=f(x, y).
R(x, y) in the expression is referred to as “image restoration filter”. A real image contains a noise component, and hence use of an image restoration filter created by taking a completely inverse number of the optical transfer function (OTF) as described above results in amplification of the noise component together with the deteriorated image. Therefore, generally, a good image cannot be acquired. In this regard, there is known a method such as use of a Wiener filter for suppressing a high frequency side restoration rate of an image according to an intensity ratio of an image signal to a noise signal. Deterioration of the image due to the color blur component is substantially corrected by, for example, causing blur amounts of respective color components to be uniform by the image blur component correction.
The optical transfer function (OTF) fluctuates according to conditions of the optical system such as a focal length (zoom state) and an aperture diameter, and hence the image restoration filter used for the image restoration processing needs to be changed.
Further, an image may contain a distortion component. The distortion is generally geometric distortion in which expansion or contraction of a peripheral part of the image is greater and which is caused by distortion aberration of an optical system.
Japanese Patent Laid-Open No. 2006-270918 discloses a method for correcting geometric distortion of an image caused by distortion aberration. The method corrects the geometric distortion by obtaining information on image pickup conditions such as a zoom state and an object distance in image pickup, and using data corresponding to the obtained image pickup condition information read from distortion aberration data prepared beforehand. The distortion aberration data is a function based on an image height. In distortion component correction, a large distortion amount of an optical system needs a large geometric transformation amount, and especially in a peripheral part of the image, scaling and transformation are performed as the geometric transformation.
In order to obtain a high-quality image by properly correcting an image deteriorated by various aberrations of the optical system, processing for reducing the image blur component and the distortion component needs to be performed.
Moreover, data of the image restoration filter used for correcting the image blur component and data of a geometric transformation condition used for correcting the distortion component need to be changed according to the image pickup conditions. The image pickup conditions include many parameters such as a zoom position, an aperture diameter and an object distance, and preparing the image restoration filter data and the geometric transformation condition data according to combinations of such many parameters significantly increases a data volume. In particular, since the image restoration filter is two-dimensional filter, the number of cells (taps) of the filter increases as the image blur component increases.
The inventor found the following problem relating to the number of cells of the image restoration filter in the correction of the distortion component with the correction of the image blur component. The distortion component correction on a deteriorated image geometrically performs scaling and transformation of the image according to an image height. On the other hand, the optical transfer function (OTF) for designing the image restoration filter can be derived from designed values or measured values of the optical system. However, in any case, the optical transfer function is derived from the optical system containing the distortion aberration.
Therefore, to perform the image restoration processing on the image on which the distortion correction processing has been performed, it is necessary to perform on the image restoration filter the scaling and the transformation according to the geometric transformation condition which has been used in the distortion correction processing. When the optical system has negative distortion, scaling, transformation and pixel interpolation are performed on the image in the distortion correction processing, so that scaling, transformation and cell interpolation are also performed therewith on the image restoration filter. As a result, the number of cells of the image restoration filter is increased, which increases the data volume of the image restoration filter. In addition, a calculation amount of filtering is also increased, which remarkably decreases a processing speed.
In a method for correcting the distortion component disclosed in Japanese Patent Laid-Open No. 2006-270918, geometric transformation correction is performed using aberration information corresponding to the image pickup condition. However, this method cannot correct the image blur component other than the distortion component, so that a high quality image cannot be obtained.